Semantic Image Synthesis via Adversarial Learning

In this paper, we propose a way of synthesizing realistic images directly with natural language description, which has many useful applications, e.g. intelligent image manipulation. We attempt to accomplish such synthesis: given a source image and a target text description, our model synthesizes images to meet two requirements: 1) being realistic while matching the target text description; 2) maintaining other image features that are irrelevant to the text description. The model should be able to disentangle the semantic information from the two modalities (image and text), and generate new images from the combined semantics. To achieve this, we proposed an end-to-end neural architecture that leverages adversarial learning to automatically learn implicit loss functions, which are optimized to fulfill the aforementioned two requirements. We have evaluated our model by conducting experiments on Caltech-200 bird dataset and Oxford-102 flower dataset, and have demonstrated that our model is capable of synthesizing realistic images that match the given descriptions, while still maintain other features of original images.Comment: Accepted to ICCV 201

TensorLayer: A Versatile Library for Efficient Deep Learning Development

Deep learning has enabled major advances in the fields of computer vision, natural language processing, and multimedia among many others. Developing a deep learning system is arduous and complex, as it involves constructing neural network architectures, managing training/trained models, tuning optimization process, preprocessing and organizing data, etc. TensorLayer is a versatile Python library that aims at helping researchers and engineers efficiently develop deep learning systems. It offers rich abstractions for neural networks, model and data management, and parallel workflow mechanism. While boosting efficiency, TensorLayer maintains both performance and scalability. TensorLayer was released in September 2016 on GitHub, and has helped people from academia and industry develop real-world applications of deep learning.Comment: ACM Multimedia 201

Topology Distance: A Topology-Based Approach For Evaluating Generative Adversarial Networks

Automatic evaluation of the goodness of Generative Adversarial Networks (GANs) has been a challenge for the field of machine learning. In this work, we propose a distance complementary to existing measures: Topology Distance (TD), the main idea behind which is to compare the geometric and topological features of the latent manifold of real data with those of generated data. More specifically, we build Vietoris-Rips complex on image features, and define TD based on the differences in persistent-homology groups of the two manifolds. We compare TD with the most commonly used and relevant measures in the field, including Inception Score (IS), Frechet Inception Distance (FID), Kernel Inception Distance (KID) and Geometry Score (GS), in a range of experiments on various datasets. We demonstrate the unique advantage and superiority of our proposed approach over the aforementioned metrics. A combination of our empirical results and the theoretical argument we propose in favour of TD, strongly supports the claim that TD is a powerful candidate metric that researchers can employ when aiming to automatically evaluate the goodness of GANs' learning.Comment: Submitted to ICML 2020; 12 pages, 7 figure

Meta-simulation for the Automated Design of Synthetic Overhead Imagery

The use of synthetic (or simulated) data for training machine learning models has grown rapidly in recent years. Synthetic data can often be generated much faster and more cheaply than its real-world counterpart. One challenge of using synthetic imagery however is scene design: e.g., the choice of content and its features and spatial arrangement. To be effective, this design must not only be realistic, but appropriate for the target domain, which (by assumption) is unlabeled. In this work, we propose an approach to automatically choose the design of synthetic imagery based upon unlabeled real-world imagery. Our approach, termed Neural-Adjoint Meta-Simulation (NAMS), builds upon the seminal recent meta-simulation approaches. In contrast to the current state-of-the-art methods, our approach can be pre-trained once offline, and then provides fast design inference for new target imagery. Using both synthetic and real-world problems, we show that NAMS infers synthetic designs that match both the in-domain and out-of-domain target imagery, and that training segmentation models with NAMS-designed imagery yields superior results compared to na\"ive randomized designs and state-of-the-art meta-simulation methods

Parametric study of density wave instability in parallel channels of a water-cooled blanket in a fusion reactor

In fusion reactors, many blanket concepts are designed with water as a coolant to transfer high-density heat from the fusion reaction out of the reactor core. The coolant temperature and pressure are maintained as the validated use in water-cooled fission reactors. However, the flow channel in a water-cooled blanket is independent of each other, and there is no flow mixing between coolant channels. Therefore, flow instability may occur in the independent parallel channels in a water-cooled blanket due to its unique structure and heat distribution, especially under the high heat flux caused by plasma rupture. In this study, the parametric analysis of density wave instability is performed using a thermal-hydraulic code developed for independent parallel channels based on the homogeneous model for the two-phase flow. The parallel-channel system in a water-cooled ceramic breeder (WCCB) blanket of the China Fusion Engineering Experimental Reactor (CFETR) is established for its first wall structure. A small disturbance is introduced into the system to determine if it is stable under different conditions. It is found that the channel number has no obvious influence on the prediction of the flow instability boundary. Therefore, the two-channel system is adopted to investigate the influence of different parameters, such as the pressure, resistance, flow rate, and inclination, on the flow instability boundary of the parallel-channel system in the CFETR WCCB blanket. The results show that flow instability occurs more easily in this study compared to the traditional instability analysis, especially under high-pressure conditions. In general, conditions of high pressure, large flow rate, and no inclination can stabilize the system, while the influence of resistance is quite different under different conditions of resistance and pressure. The research work indicates that more attention should be paid to the joint influence of different parameters for the water-cooled blanket during its design and operation

DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction

Compressed Sensing Magnetic Resonance Imaging (CS-MRI) enables fast acquisition, which is highly desirable for numerous clinical applications. This can not only reduce the scanning cost and ease patient burden, but also potentially reduce motion artefacts and the effect of contrast washout, thus yielding better image quality. Different from parallel imaging based fast MRI, which utilises multiple coils to simultaneously receive MR signals, CS-MRI breaks the Nyquist-Shannon sampling barrier to reconstruct MRI images with much less required raw data. This paper provides a deep learning based strategy for reconstruction of CS-MRI, and bridges a substantial gap between conventional non-learning methods working only on data from a single image, and prior knowledge from large training datasets. In particular, a novel conditional Generative Adversarial Networks-based model (DAGAN) is proposed to reconstruct CS-MRI. In our DAGAN architecture, we have designed a refinement learning method to stabilise our U-Net based generator, which provides an endto-end network to reduce aliasing artefacts. To better preserve texture and edges in the reconstruction, we have coupled the adversarial loss with an innovative content loss. In addition, we incorporate frequency domain information to enforce similarity in both the image and frequency domains. We have performed comprehensive comparison studies with both conventional CSMRI reconstruction methods and newly investigated deep learning approaches. Compared to these methods, our DAGAN method provides superior reconstruction with preserved perceptual image details. Furthermore, each image is reconstructed in about 5 ms, which is suitable for real-time processing

Radiotherapy in the preoperative neoadjuvant treatment of locally advanced rectal cancer

Radiotherapy and chemotherapy are effective treatments for patients with locally advanced rectal cancer (LARC) and can significantly improve the likelihood of R0 resection. Radiotherapy can be used as a local treatment to reduce the size of the tumor, improve the success rate of surgery and reduce the residual cancer cells after surgery. Early chemotherapy can also downgrade the tumor and eliminate micrometastases throughout the body, reducing the risk of recurrence and metastasis. The advent of neoadjuvant concurrent radiotherapy (nCRT) and total neoadjuvant treatment (TNT) has brought substantial clinical benefits to patients with LARC. Even so, given increasing demand for organ preservation and quality of life and the disease becoming increasingly younger in its incidence profile, there is a need to further explore new neoadjuvant treatment options to further improve tumor remission rates and provide other opportunities for patients to choose watch-and-wait (W&W) strategies that avoid surgery. Targeted drugs and immunologic agents (ICIs) have shown good efficacy in patients with advanced rectal cancer but have not been commonly used in neoadjuvant therapy for patients with LARC. In this paper, we review several aspects of neoadjuvant therapy, including radiation therapy and chemotherapy drugs, immune drugs and targeted drugs used in combination with neoadjuvant therapy, with the aim of providing direction and thoughtful perspectives for LARC clinical treatment and research trials